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Abstract. The paper is concerned with a general optimization problem for a nonlinear control system, in 
the presence of a running cost and a terminal cost, with free terminal time. We prove the existence of a 
patchy feedback whose trajectories are all nearly optimal solutions, with pre-assigned accuracy. 



1 - Introduction 

Consider a general optimization problem 



mm 

T,«(-) 



i^{x{T))+ r L{x{t),u{t))dt\ , (1.1) 



for a nonlinear control system of the form 

x = f{x,u) u{t)e\J. (1.2) 

Here x € describes the state of the system, the upper dot denotes a derivative w.r.t. time, and U C IR™ 
is the set of admissible control values. The minimum is sought over all times T > and all measurable 
control functions u : [0, T] i-^ U. 

In the literature, several results are available, which provide the existence of an optimal control t i— > 
u°P*{t) in open-loop form [14, 16, 23], for any fixed initial condition 

a;(0) = ye-K". (1.3) 

On the other hand, the existence and regularity of an optimal control in feedback form is a far more difficult 
issue. In an ideal situation, one would like to construct a (sufficiently regular) feedback u — U{x) such that 
all trajectories of the corresponding O.D.E. 

x^f{x,U[x)) (1.4) 

are optimal w.r.t. the cost criterion (1.1). Only few general results are presently known in this direction [7, 
16, 20, 26]. In general, the optimal feedback can be discontinuous, with an extremely complicated structure 



[8, 17]. Moreover-, its performance may not be robust: an arbitrarily small external perturbation may produce 
trajectories which are far from being optimal [24]. 

An alternative strategy, pursued in [3, 18, 19], is to construct sub-optimal feedbacks, trading off the full 
optimality in favor of a simpler structure of the control and the robustness of the resulting system. This ap- 
proach also faces difficulties. In some cases, because of topological obstructions it is not possible to construct 
any continuous asymptotically stabilizing feedback [10, 13, 14, 25], or any continuous near-optimal feedback 
[9]. Therefore, one needs to work with discontinuous feedback controls [11, 12, 21. 22]. For discontinuous 
O.D.E's, however, no general result about existence and uniqueness of solutions is available. Caratheodory 
solutions can be constructed only under additional assumptions on the structure of discontinuities [15]. 

Following the approach developed in [1,2,3], asymptotic stabilization and optimal control problems 
can be solved using patchy feedbacks as discontinuous controls. We recall that a patchy feedback has a 
particularly simple structure, since it is a function u = U{x) that is piecewise constant on the state space 
iR". For patchy vector fields, one can prove that Caratheodory solutions forward in time always exist [1]. 
Moreover, the set of forward solutions is stable w.r.t. small perturbations [2]. The analysis in [3] showed 
that any minimum time problem can be approximately solved using these patchy feedbacks. 

Aim of the present paper is to extend the results in [3] to the general optimization problem (1.1). In 
addition, we present a construction which greatly simplifies the previous approach, thus clarifying the main 
lines of the proof. 

For convenience, we list here all the basic assumptions. 

(A) The set of admissible control values U C JR™ is a compact, the function / : J?" x J7 i— > JR" is continuous 
w.r.t. both variables, and twice continuously differentiable w.r.t. x. In addition, / satisfies the sub-linear 
growth condition 

|/(a;,u)| < C/(l + |a;|) for all uSU, (1.5) 

for some constant C/ . Both the terminal cost tp : i-^ M and the running cost L : xXJ i—^ M are 
continuous and non-negative. Moreover, L is strictly positive: 

L{x, m) > ao > for all x & ST , w € U . (1.6) 

Throughout this paper, V denotes the value function for the optimization problem (1.1)-(1.2), namely 

V{y) = inf U{x{T)) + f L{;x{t), u{t)) dt \ , (1.7) 

where the minimization is taken over all T > and all solutions of 1 1— > x{t, u), corresponding to a measurable 
control u : [0, T] t-^ U. Our main result can be stated as follows. 

Theorem 1. Let the functions ipyL,/ in (1.1)-(1.2) satisfy the assumptions (A). Let e > and a compact 
set K C he given. Then there exist a closed terminal set S C JR" and a patchy feedback u = U (x) defined 
on the complement iR" \ S such that the following holds. For each y G K, every Caratheodory solution of 

X = f{x, U{x)) , x{0) = y (1.8) 

reaches the set S within finite time. Calling t = inf {t ; x{t) G 5} the first time where the trajectory reaches 

S, we have 

^P{x{t))+ [ L{x{t),U{x{t)))dt <V{y) + e. (1.9) 
Jo 

We recall that, by well known properties of patchy vector fields, for every initial point y € JR" \ S the 
O.D.E. (1.8) has at least one forward Caratheodory solution. According to (1.9), all of the solutions starting 
from the compact set K are nearly optimal, for the cost (1.1). 
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In the remainder of the paper, Section 2 contains a brief review of the main definitions and properties 
of patchy feedbacks and patchy vector fields. The proof of Theorem 1 is then worked out in Section 3. 



2 - Review of patchy feedbacks 

The following definitions were introduced in [1]. 

Definition 1. By a patcli we mean a pair (yi, g) where C JR" is an open domain with smooth boundary 
and g\s& Lipschitz continuous vector field defined on a neighborhood of the closure ^1 of fl, which points 
strictly inward at each boundary point x G dfl. 

Calling n(a;) the outer normal at the boundary point x, and denoting the inner product by a dot, we 
thus require 

n(a;) • g{x) < for all x G dQ. (2.1) 

Definition 2. We say that g : il. i-^ JR" is a patchy vector field on the open domain if there exists a 
family of patches {(Oc, ga); a G A} such that 

- ^ is a totally ordered set of indices, 

- the open sets fla form a locally finite covering of 11, 

- the vector field g can be written in the form 

g{x) = ga{x) if a; € \ |J ^f3- (2.2) 

/3>a 

We shall occasionally adopt the longer notation (f], g, {fla, ffa)„e^) to indicate a patchy vector field, 
specifying both the domain and the single patches. 

By setting 

a*{x) = max^a £ A ; xGfla}, (2.3) 
we can write (2.2) in the equivalent form 

aix) = 9,.(,)(a;) for all i a; e O. (2.4) 

Remark 1. It is important to observe that the patches (17^, ga) are not uniquely determined by a patchy 
vector field {il, g). Indeed, whenever a < /3, by (2.2) the values of ga on the set fi^ n are irrelevant. Of 
course, the values of ga for x outside the domain Q. don't matter either. Therefore, if the open sets form 
a locally finite covering of Q. and if for each a & A the vector field ga satisfies 

T^a{x) ■ ga{x) < for all x S O H di^a \ \^ ^0 , (2-5) 

/3>a 

then the vector field g in (2.2) is still a patchy vector field. Indeed, without changing the function g, one can 
suitably redefine the values of each ga on the set U/3>a outside fl, and achieve the strict inequality 

^a{x) ■ ga{x) < for all X S dQ,a ■ 

Remark 2. For convenience, we are always assuming that the single patches fi^ are open, while the vector 
fields ga are defined on the closure Q.a ■ In certain situations, it would be natural to choose patches of the 
form 

f2i = {a;Gf2; n • a; < c} , f22 = {a;SO; n • a; > c} , 
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for some unit vector n. In this way, however, the union Oi U fl2 does not cover all of Q, because it does not 
contain points where n - x = c. This situation is easily fixed, replacing fli by a slightly larger open set which 
contains also these boundary points. The resulting vector field 



can still be written in patchy form. 



gi{x) if n • X < c , 

g2{x) if n - X > c, 



If fif is a patchy vector field, the differential equation 

x = g{x) (2.6) 

has several useful properties. There are collected in the following theorem, proved in [1]. 

Theorem 2. Let g he a patchy vector field. Then the set of Caratheodory solutions of (2.6) is closed (in the 
topology of uniform convergence) but possibly not connected. For each Caratheodory solution t i— > x{t) , the 
map 1 1— > a*{x{t)) defined by (2.3) is left- continuous and non- decreasing. 
Given an initial condition 

x{to) = xo , (2.7) 

the Cauchy problem (2.6)- (2.7) has at least one forward solution and at most one backward solution, in the 
Caratheodory sense. 

Remark 3. In some situations it is convenient to adopt a more general definition of patchy vector field than 
the one formulated above. Indeed, one can consider patches {fla, go) where the boundary of the domain 
J7q is only picccwise smooth. For example, could be a polytope, or the intersection between a ball and 
finitely many half-spaces. In this more general case, the inward-pointing condition (2.1) can be reformulated 
by asking that, for each boundary point x € dO.^, the vector ga{x) lies in the the interior of the tangent 
cone to at the point x. Namely 

5«(a;) SintTnJx). (2.8) 

As in [4], this tangent cone is defined by 

, , f d(x + tv, r^ct) 1 

Tnix) = {v&M' : liminf — ^ = L 

°' ' [ no t J 

One can easily check that all the results concerning patchy vector fields stated in Theorem 2 remain valid 
with this more general formulation. 

Definition 3. Let (fi, g, (0^, 5a)„e^) be a patchy vector field. Assume that there exist control values 
€ U such that, for each a & A, there holds 

9oi{x) = f{x, Va) for ah x G \ |J % • (2.9) 

f3>a 

Then the piecewise constant map 

U{x) =Voc if X & Vto,\[J n^. (2.10) 

I3>a 

is called a patchy feedback control on Q. . 

Recalling (2.3), the patchy feedback control can thus be written on the form 
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3 - Proof of the theorem 



The proof of Theorem 1 will be given in several steps. 

1. Various reductions can be performed. By a smooth approximation, we can assume that tp G C°°. 
Moreover, approximating the cost function L by a more regular function, it is not restrictive to assume that 
L is twice continuously differentiable w.r.t. x. Recalling that L{x,u) > ccq > 0, we can now replace f{x,u) 
by 



and consider the equivalent problem 



with dynamics 



inf |r + V^(x(T))|, (3.2) 



x = g{x,u), x{0) = y. 

Notice that the function g in (3.1) is continuous w.r.t. both variables x,u, and twice continuously differen- 
tiable w.r.t. X. Moreover it satisfies the growth condition 

\g(x,u)\ <^ (l + \x\) for all usU. 

In the following, we thus assume without loss of generality that the running cost is simply L{x,u) = 1, so 
that the minimization problem (1.1) reduces to (3.2). 



2. Choose a constant M such that 



M > 1 , M> max ip{x) . (3.3) 



To fix the ideas, throughout the following we assume that < e < 1/8 and that the compact set K 

is contained in the open ball Bp centered at the origin with radius p. Because of the sub-linear growth 
condition (1.5), for r < 2M, every trajectory of the system (1.2) starting at a point y € K C Bp will satisfy 
the a priori bound 

\x{t)\ < p for all te[0,T] C [0,2M], (3.4) 

where 

p = e^'f^^{p+l). (3.5) 



3. Let V = V{y) be the value function for the optimization problem (3.2), with dynamics (1.2). We claim 
that V is semi-concave. More precisely, there exists a constant k such that, for any y,y' G Bp, one has 

V{y')<V{y)+w.{y'-y) + K^y-^. (3.6) 

for some vector w G D~^V{y) in the upper gradient of V at the point y. 

Indeed, from the theory of optimal control [6] it is well known that the optimization problem (3.2), 
(1.2) with initial data x{0) = y has at least one solution, within the class of chattering controls. Let 
1 1— > x(t) = x{t ; y, u, 9) be an optimal chattering trajectory, with 

n 

ar(0)=y, x{t) =Y,Ht) f{<t)Mt)) t&[Q,T], (3.7) 

i=0 

for some measurable functions (u, 6) = {uq, . . . , u„, Oq,..., On) satisfying 

n 

Mi:[0,r]^U, : [0,r] ^ [0,1], ^0,(t) = l. (3.8) 
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For any other initial data y' . wo can consider the same chattering control (u, 9), always stopping at the same 
terminal time t = t. This yields the cost 



y»A-(y')=T + V'Hr;y',n,^)). 



(3.9) 



The regularity assumptions on /,■)/) w.r.t. the variable x imply that, as y' varies in the ball Bp, the map 
y' h- > y">^'^(y') is twice continuously differentiable. Moreover, its norm remains bounded: 



|T/-M,e,T|| ^ 

r \\e^(Bj,)^'^- 



(3.10) 



Since r e [0, Tmax] while both u and 6 in (3.8) range over compact sets, this bound is uniform, i.e. in (3.10) 
wc can take a constant k> 1 which does not depend on the particular chattering control, or on the time r. 
Observing that 



V{y) = V^'^'^y) , V{y') < F«'^'-(t/') 

the inequality (3.6) follows from (3.10), choosing w = Vy"'^''^(y). 
4. As shown in the previous step, the value function 

V{y) = min F"'^"'^(y) 



for all y' e Bp. 



is Lipschitz continuous on the ball Bp. In fact, the constant k > 1 in (3.10) also provides a Lipschitz constant 
for V, namely 

V{x) -V{y) < K\x-y\ for all x,y e Bp . (3.11) 

By Rademacher's theorem, V is differentiable almost everywhere. At each point x € Bp where the 
gradient W{x) exists, if V{x) < tlj{x) then one has the well known relation [5, 10, 16] 



Consider the open set 



min {\/V{x) ■ fix, m)} + 1 = . 



V={x; V{x) <tjj{x)} . 



(3.12) 



Given 5 > 0, we can choose finitely many points yi,. . .ym € BpCiV such that W{yi) is well defined for 
each i = 1, . . . , to, and moreover 



BpnVc\jB{yi,d). 

Define the approximate value function 

W{x)=m.in {^{x), Wi{x), ... , W„(x)} , 

where 



(3.13) 

(3.14) 
(3.15) 



W,{x) = V{y,) + VV{y,) ■ (x - y,) + k \x - y,f. 
We claim that, by choosing S > sufficiently small, for all x € Bp the following relations hold. 

V{x) <W{x) <V{x) + e, (3.16) 



min {VWi{x)- f{x,u)} + l 



< e 



whenever Wi{x) = W{x) . 



(3.17) 
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Indeed, the first inequality in (3.16) follows from (3.6). Next, since / is continuous and U is compact, 
we can find 6i G ]0, 1] such that the following conditions hold. If x € Bp , w = W{y) exists and 

min {w • f{y,u)} + 1 = 0, 
|w' — w| < 2k,Si , \x — y\ < Si , 

then 

min i w • f{x,u)} + l\ <e. (3.18) 

We now choose 5 > such that 

2kS + kS"^ < min |e , -^^j . 

Given any x G Bp, if j is an index such that \x — yj\ < 6, recalling the Lipschitz condition (3.11) we find 
W{x) < V{yj) + \S7V{yj)\\x-yj\+K\x-yj\^ < V{x) + 2k\x - yj\ + k\x - yj\^ , 

W{x)-V{x) < min {e, (3.19) 
This already yields (3.16). Comparing (3.6) with (3.15) we notice that 

Wi{x)-V{x)>K^-^^^^. 

Hence from (3.19) it follows 

\x-yi\<6i, whenever Wi{x) = W{x) . (3.20) 

Observing that, if W{x) = Wi{x), 

\WWi{x)-VWi{yi)\ < 2K\x-yi\ < 2k6u 
from (3.18) we deduce the inequality (3.17). This establishes our claim. 

5. By the definition of Wj, it is clear that all level sets where Wi is constant are spheres. Indeed, for any 
given constant c we can write 

{x; Wi{x)^c} = [x; \x - Xi\ = r} , 

with Xi = yi — W{yi)/2n and a suitable radius r. 
For each i= 1, . . . , m, consider the set 

Vi = {x&Bp- Wi{x)=W{x)). (3.21) 

In this step we show that there exists a minimum radius Tmin > and a maximum radius Vmax such that, 
fixed x &Vi, the level set where Wi = Wi{x) is a sphere of center and radius r with 

fmin ^ ^ '^max • (3.22) 

Indeed, since e < 1/2, by (3.17) it follows 

\VWi{x)\\f{x,u)\>l-. (3.23) 



Calling 

from (3.23) we deduce 



Mf = max \f{x,u)\<Cf{l+p), 
|x|<p,ueu 



|Vm(x)| > ^ 



/ 



Therefore, for any ^ such that Wj(^) = Wi{x), 



On the other hand, by (3.15) and (3.20) we have 

\VWi{x)\ < \VWi{yi)\ + 2k\x - yi\ < K + 2Kdi < 3k. 
Hence, for any ^ such that Wi(^) = Wi{x), 

\^ - Xi\ = \X - Xi\ = ^^^^^^^^ ^ ^^'^^^ 



6. We are now ready to construct the near-optimal patchy feedback. We will define U{x) on the open set 

n = {xGBp; W{x) < ip{x)} , (3.25) 

and the required terminal set 5 will be defined as 5 = JR" \ O. Given t] > small, for each point x G Vi 
consider the point (see Figure 1) 

2 1 X — Xi 

and the ball Bf = B(p^, \x — Xi\/3) centered at with radius r = \x — Xi\/3 . By (3.17), there exists a 
nearly-optimal control value u = uf £\J such that 

VWi{x)-f{x,u^)<-l + s. (3.26) 

Consider the lens-shaped region 

Tf = B (pt , \B{xi, \x-xi\-v). (3.27) 
Its upper boundary will be denoted as 

d+Tf = dr^ \ B{xi , \x -x,\-7j). (3.28) 
Moreover, for z G d~^Tf, we write ni{z) for the outer unit normal at the point z. 
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We claim that, by choosing r/ > sufBciently small, the following holds: 

VWi(0) ■ f{z, u^) <-l + 2s for all z €^ , (3.29) 

n^{z) ■ f{z, O < ~T] for all z E d+T^ . (3.30) 

Moreover, the constant 77 > can be chosen uniformly valid for all i = 1, . . . , m and all x ^T^i . 

For fixed i, x this is clear because, as 77 — > 0, the diameter of the set approaches zero. Moreover, as 
z varies on the upper boundary , all the unit normals Ta.i{z) approach the vector VWi(a;)/| VWi(a;)| . 

Therefore, both inequalities (3.29)-(3.30) follow from (3.26). 

We now observe that / = f{x, u) is uniformly continuous on the compact domain Bp x U. Moreover, on 
each set 2?^, the gradient VVFi(x) is uniformly Lipschitz continuous and bounded away from zero. Finally, 
the radius of each level set, where Wi is constant, by (3.22) is uniformly bounded above and below. This 
allows us to choose a constant 77 > uniformly valid for all i, x. 

7. To achieve a nearly optimal feedback, we would need the inequality 

VW{z) ■ f{z, Ui) < -1 + 4£ for all z G . (3.31) 

If W{z) = Wi{z) for all z G Tf, this is a trivial consequence of (3.29). However, we must also consider the 
case where some of the points z €Tf lie in a region where W{z) = Wj{z) < Wi{z), for some different index 
j. For this purpose, we observe that the set where Wi = Wj is always a hyperplane, say 

H^j ={x: W,{x) = Wj {x)] = {x- n,j ■ x = } . (3.32) 

for a suitable constant Cy and a unit normal vector riy . The orientation of will be chosen so that 

[x ; Wi{x) < Wj{x)] = {x ; riy ■ x < Cij] . 

We claim that, by choosing 77 > suflaciently small, uniformly w.r.t. i,x, one of the following two cases 
occurs (see Figure 2). 

CASE 1: At every point 2; G F? n Hij one has 

ni,-./(2,<) < -77. (3.33) 

CASE 2: At every point -2 G F? one has 

VVF,(0)-/(z,<) < -l + 4£. (3.34) 

Indeed, assume that (3.33) fails. Then there exists a point 2* G F? n T-Lij such that 

n,,-!iz\v;[) > -r^. (3.35) 
By (3.32) and the orientation of the unit vector rijj , we can write 

VWj{z*) = VWi{z*) -(3nij (3.36) 
for some constant /3 > 0. Together, (3.29) and (3.35) now imply 



WWj{z*) ■ f{z\u^) = VW,{z*) • /(z*,<) - /3n,,- • /(z*,<) 
< -l + 2£ + /377 < -l + 3e. 



(3.37) 



provided that we choose 77 > sufficiently small. Since / and VWj are uniformly Lipschitz continuous, from 
(3.37) it follows that (3.34) is valid for all z sufiiciently close to z*. By reducing the size of 77 > 0, we can 



w>w \ w<w. 

i j ' 1 J 

Figure 2. If the domain Tf intersects the half-space where Wj < Wi, two cases must be considered. 
Left: in Case 1, the vector field f{-,uf) points toward the set where Wi < Wj. As a patch we then 
take the shaded region CTf. Right: In Case 2, f{-,uf ) points toward the set where Wj < Wi. 
We can now take Clf =Tf, because the control uf is nearly optimal on this whole region. 

make the diameter of the lens-shaped domain Tf as small as we like. Hence the inequality (3.34) will hold 

for all z eT^ . 

To prove our claim, it remains to observe that the functions / and VWj are uniformly continuous, and 
that the constant /? in (3.36) remains uniformly bounded. Hence the constant 77 > can be chosen uniformly 
valid for alH, j, a; . 

We now define the smaller domain 

nf = Tn\J{z€M"; Wjiz) < Wiiz)} (3.38) 

where li C {1, . . . , m} is the set of indices j ^ i for which CASE 1 applies. 

By the previous analysis, for each j such that W{z) = Wj{z) for some z €Tf, two cases can occur. If 
CASE 1 applies, then the vector field f{-,uf) is strictly inward-pointing along the portion of the boundary 
dflf where Wi = Wj. On the other hand, if CASE 2 applies, then (3.34) holds on the entire domain Tf . 

8. Consider the family of all domains flf, as i G {!,..., m} and x ranges over the closure of the set 
= {x £ Bp-, W{x) < tp{x)}- It now remains to select finitely many domains Of which cover the compact 
set fl. This last step, however, must be done with some care because on the lower portion of the boundary 

d-nf = dnf nB{xi, \x-Xi\-ri) (3.39) 

the vector field f{-,Ui) may not be inward-pointing. To cope with this problem, we first observe that there 
exists a uniform constant h> such that 

Wi{z)<Wi{x)-h, (3.40) 

for every i, x and every z € d~flf . 

We now set M* = max {W^(a;) ; x £ Bp}, and split the domain O in sub-domains of the form 

ni = {xen; M* -{£+l)h<W{x) <M* -ih}. (3.41) 

For each £, wc cover the; compact set f2^ with finitely many domains Qf, constructed as in step 7, choosing 
X e . After a relabelling of both the domains and the correspondent vector fields from (3.26), this yields 
the patches (see Figure 3) 

{fle,a, f{;Ui,a)) , a = l,...,Ne. (3.42) 
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Figure 3. The domain = U is covered by a family of patches ^i^a , ordered like tiles on a roof. 



On the collection of all patches (3.42) we define the lexicographic order: 

{£, a) -< {£', a') iff either £ < £' or £ = and a < a'. 
We claim that the above construction yields a patchy vector field: 

g{x) = f{x,Ue,a) ifi" XGflt^a\ [J ^£',a' (3.43) 

Indeed, according to Remark 1, it suffices to check that, for each patch ^le^a — ) the vector field /(•, u^,a) = 
/(•, uf) is inward pointing at every point of the set 

In the present case, this is clear, because the only boundary points where f{-,uf) is not inward pointing are 
those on the lower boundary d~Qf. Since x £ Q( we have W{x) < M* — £h, and hence by (3.40) 

W{z) <M*-{£+ l)h for all z e d-flf . 

Therefore, given any point z G d~flf , either W{z) = tp{z) and ^; ^ O, or else z is contained in a patch flc^a' 
with £' > £, as required in Remark 1. 

9. To complete the proof, we now check that the patchy feedback that we have constructed is nearly optimal. 
We recall that, by the analysis in step 7, for every i, x we have 

VWi{z) ■ f{z, O < -1 + 4e for all ^ G il^ (3.44) 

Now take any initial point y & K and let t x{t) be any Caratheodory solution of the Cauchy problem 

x = g{x) x{0)=y, 

with g defined at (3.43). li y G K \n, Q = {x € Bp-, W{x) < ipix)} as in (3.25), we are in the terminal 

set S and there will be no evolution, since it is more convenient to stay in y than to move along a trajectory. 
Otherwise, call r > the first time at which x{t) reaches the boundary of the set f2. By (3.44) we have 

W{x{T))^W{y) = £ [j^W{^it))]dt < (-l + 4£)r, 

hence 

r < W^y)-n<r)) ^ ^^^^^ ^ 2^^^^ ^2M. 
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By (3.4), it follows that x{t) cannot be on the boundary of Bp. Wc thus conclude that W(^x{t)) = ■i/'(a;(r)). 
Stopping at time r, since W{x) > and V{y) < M, the total cost can be estimated as 

r + i>{x{T)) < ^— +'<P[x{t)) < -Ynr - ^ 1 - £ • 

Since £ > was arbitrary, this completes the proof. 
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